Joerg Jaspert: HTML output for packages in NEW
You may remember that, short before the release,
Aj had a HTML page for
packages in NEW, living in his ~/ajt/new/ on our ftp-master host.
It was based on a
patch
from Thomas Viehmann and got lost with the ries crash.
Today I reimplemented it, putting it directly into dak and the
ftp-master codebase and webtree. My work is also based on Thomas patch,
but I reworked the whole thing a bit. I’ve written a new dak command,
show-new, which is based on process-new, the command I always use when I
process NEW packages. Obviously not with all that functionality, it
doesn’t need any reject capability, but it still has some nice
parts. Like
- it only runs on packages and their versions if no html output had been generated in the past
- Automatically removes html files of packages no longer in NEW
- It runs right after generating the NEW overview page, hourly. So it should be similar up-to-date (or outdated) as that page is.
2007-12-30 Joerg Jaspert joerg@debian.org> * dak/dak.py (init): add show-new. This is based on a patch submitted by Thomas Viehmann in Bug #408318, but large parts of handling it are rewritten and show-new is done by me. * dak/queue_report.py (table_row): Add link to generated html page for NEW package. * dak/show_new.py: new file, generates html overview for NEW packages, similar to what we see with examine-package. * config/debian/cron.hourly: Add show-new call * config/debian/dak.conf: Add HTMLPath for Show-New * dak/examine_package.py (print_copyright): ignore stderr when finding copyright file. (main): add html option (html_escape): new function (escape_if_needed): ditto (headline): ditto (colour_output): ditto (print_escaped_text): ditto (print_formatted_text): ditto - use those functions everywhere where we generate output, as they easily know if we want html or not and just DTRT (do_lintian): new function (check_deb): use it (output_deb_info): Use print_escaped_text, not print_formatted_text. Also import daklib.queue, determine_new now lives there Also add a variable to see if we want html output. Default is disabled, show_new enables it for its use. Most of html, besides header/footer are in examine_package instead of show_new, as it makes it a whole lot easier to deal with it at the point the info is generated. * dak/process_new.py (determine_new): Moved out of here. (check_valid): Moved out of here. (get_type): Moved out of here. * daklib/queue.py (determine_new): Moved here. (check_valid): Moved here. (get_type): Moved here. * dak/init_db.py (do_section): Remove non-US code * dak/make_overrides.py (main): ditto * dak/process_new.py (determine_new): ditto * daklib/queue.py (Upload.in_override_p), (Upload.check_override): ditto * daklib/utils.py (extract_component_from_section):, (poolify): ditto * dak/import_archive.py (update_section): ditto * dak/symlink_dists.py (fix_component_section): ditto * scripts/debian/mkmaintainers: ditto * scripts/debian/update-mirrorlists (masterlist): ditto * config/debian-non-US/*: Remove subdir * scripts/debian/update-readmenonus: Removed.In the recent past I did some more dak changes. Just to list them here, they are
- Fixing daks handling of keys that have expired subkeys together with non-expired subkeys (or non-expired “master” key). That created silent rejects (only sent to ftpmaster, not to the maintainer) in the past. Now it works, iff the signature is valid. (We check other reject reasons like unknown or revoked key, badsig and some other things that can go wrong before that).
- Internal for the NEW queue I adjusted the lintian call to also show what lintian warnings/errors maintainers try to override - and to have its display in color. lintian now also runs on the .dsc file, not only on the individual changes, which also helps me to spot some more errors in NEW which I previously may have missed
- The script to look at the packages in NEW also had a bug that disallowed to look at copyright files from packages using bzip2 compression in their binary tarballs. It did use some ar p tar xz pipeline, which I changed to using dpkg-deb -x and —fsys-tarfile.
- I removed a check for a dpkg pre-depends for bzip2 using binaries, as the dpkg version from sarge knows how to deal with them.